Methods for Stochastic Optimal Control under State Constraints

This thesis looks at a few different approaches to solving stochastic optimal control problems with state constraints. The motivating problem is optimal control of an energy buffer in a hybrid vehicle, although applications are abundant in a number of areas. Stochastic optimal control problems can be solved via the socalled Hamilton-Jacobi-Bellman (HJB) equation. State constraints result in boundary conditions for the HJB equation causing the value function to go to infinity as the state approaches the boundary, which makes it difficult to solve this partial differential equation numerically. Different approaches to avoiding infinite values on the boundary are investigated. First, we consider a logarithmic transformation of the value function. This results in an exact linearization, turning the HJB equation into an eigenvalue problem in the one-dimensional case, and also in higher dimensions, but then with certain restrictions on the relation between noise and control cost. Then, for a more general problem formulation, we introduce a different transform which yields a nonlinear problem. It is investigated under what conditions the boundary constraints will be well-behaved, and example problems are solved using a collocation method, demonstrating how a small number of collocation points is sufficient to yield good solutions in those cases. Finally, we consider a method starting from the FokkerPlanck equation. This yields an equivalent problem, but where the value function of the HJB equation need not be computed explicitly, but the probability density function of the closed-loop system is computed instead. This fact can be utilized to focus computational resources on the parts of the state-space that are the most relevant.